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Abstract: The United States Army is investing in simulations as a way of providing practice for leader decision making. Such simulations, 
grounded in lessons learned from deployment experienced leaders, place less experienced and more junior leaders in challenging situations 
they might soon be confronted with. And given increased demands on the Army to become more efficient, while maintaining acceptable levels 
of mission readiness, simulations offer a cost effective complement to live field training. So too, the design parameters of such a simulation 
can be made to reinforce specific behavior responses which teach leaders known theory and application of effective (and ineffective) decision 
making. With this in mind, the Center for Army Leadership (CAL) determined that decision-making was of critical importance. Specifically, the 
following aspects of decision-making were viewed as particularly important for today’s Army leaders: 1) Decision dilemmas, in the form of 
equally appealing or equally unappealing choices, such that there is no clear “right” or “wrong” choice 2) Making decisions with incomplete or 
ambiguous information, and 3) Predicting and experiencing second- and third-order consequences of decisions. It is decision making in such 
a setting or environment that Army leaders are increasingly confronted with given the full spectrum of military operations they must be 
prepared for. This paper details the approach and development of this decision making simulation. 


1.0 INTRODUCTION 

The ambiguous, less than perfect information 
and limitations of human perception, match 
well with the contemporary operating 
environment which US Army leaders routinely 
face. For this reason the Center for Army 
Leadership (CAL) determined decision 
making in such a situation was a high priority 
developmental need for its leaders. 
Consequently, resources were dedicated to a 
simulation that would provide leaders with 
practice in this type of decision making. A 
first step in the simulations development was 
to identify appropriate decision making 
theories, principles and models around which 
the simulation would be developed. 
Naturalistic decision making was identified 
early on as a way of thinking about decision 
making consistent with what Army leaders 
actually experience. A naturalistic framework 
emphasizes real world decision making in 
demanding situations that require cognitively 
complex thinking skills. From among the 
various models that take a naturalistic 
approach, two in particular were important to 
the development of the simulation. 


2.0 THEORETICAL APPROACH 

Snowden and Boone’s (2007) Cynefin 
Framework (See Figure 1), the first approach 
for decision-making, provided a useful way to 
think about the context for the simulation. 
The framework identifies five contexts, 
defined by the predominant cause and effect 
relationship found in each, in which a leader 
may find him- or herself: simple, 

complicated, complex, chaotic, and in- 
disorder. In keeping with the important 
aspects of decision-making we identified, our 
focus was on the ordered complicated and 
unordered complex domains. Complex 
contexts are characterized by the lack of a 
clearly right answer, unpredictability, and flux. 
Snowden and Boone argue that in such a 
context, a leader should first probe or 
investigate, then assess the facts, and finally 
respond. Such contexts often require 
creative or innovative decisions, which may 
be accompanied by a risk of failure. 
Consequently, inexperienced leaders or 
leaders who are not comfortable with the 
uncertainty of the situation may fall back to a 
more controlling style of decision-making. 
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Figure 1: Cynefin Framework 


The second theoretical model that provided 
structure to the decisions featured in the 
simulation was Quinn and Rohrbaugh’s 
(1981) competing values framework. The 
primary tenet of the model is that 
organizational effectiveness is dependent on 
meeting numerous performance criteria that 
are organized by four value sets. At its core, 
this model is based on tensions - tensions 
between values and tensions between 
choices. In applying the competing values 
framework, we attempted to create tension 
between decision options. One source of 
tension was that for many of the scenarios, 
the decision choices were equally attractive 
or equally unattractive. There were 
numerous trade-offs among the choices, so 
that no single option was clearly 
advantageous. Another source of tension 
was between immediate outcomes and long- 
term outcomes of the decisions. For 
example, for a particular choice, the 
immediate outcomes may be very positive but 
the long-term outcomes are disastrous for 
one or more parties. Alternatively, another 
choice may lead to negative immediate 
outcomes, but the long-term outcomes are 
beneficial. An added advantage to using the 
competing values framework was that it 
contributed to the realism of the decisions. 
Often, real-life decisions are not clear-cut and 
trade-offs must be weighed. 

Combining these two naturalistic approaches, 
the Mission at Mubasi simulation puts leaders 
in leadership dilemma situations which are 
initially represented as simple and ordered 
choices. As the simulation progresses 
choices become unordered and complex. 


The learner must gather data from multiple 
sources and become aware of patterns in the 
environment in order to evaluate a best 
course of action. Thereby the simulation 
progresses towards more ambiguity and the 
learner must examine problems beyond 
single variable possibilities. 


3.0 TECHNICAL APPROACH 
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Figure 2: Technical Framework 


Our initial objective was to create a simulation 
that allowed the learner to evaluate 
competing information and determine a best 
course of action. It was decided that, in order 
to constrain complexity, the framework would 
be designed to present decision dilemmas as 
discrete points in the simulation rather than 
continuous actions taken to shape the 
simulated environment. These discrete 
decision points would be multiple choice 
dialogue interactions where virtual avatars 
(one or more) would provide context to the 
decision. 


4.0 DEVELOPING MISSION AT 
MUBASI 

An implementation of Mission at Mubasi was 
created for the US Center for Army 
Leadership’s Multi-Source Assessment and 
Feedback (MSAF) program called Mission at 
Mubasi. The context of the simulation is to 
enable senior NCOs and junior officers to 
enhance their decision-making skills by 
immersing them in an engaging interactive 
game based training environment. 

Mission at Mubasi is based on a fictitious 
North African country that is in need of 
governance and infrastructure development. 
Within the game the learner plays a leader 
new to the African country situation (i.e. 
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taking over from a previous leader) and the 
environment is negative for several reasons 
including lack of support for the Army 
governance, lack of ample food and water, 
and minimal security. There are also multiple 
parties with differing, sometimes conflicting, 
priorities and interests that the leader must 
address. In the course of the simulation, the 
leader goes through a series of scenarios in 
which he or she is provided with incomplete, 
ambiguous information from multiple sources, 
and then must select an option from various 
courses of action, none of which were 
designed to be ideal responses over the 
others. 

Game Variables : Within the game the 
learner’s objective is to maintain four 
important aspects of the governance process: 
relationship and positive affect of the 
villagers, well being of his soldiers, security in 
the environment, and transitioning governing 
capacity to the local tribes. Each of these 
values will be modified during the simulation 
by decisions and actions taken by the learner. 

Conditions will arise either through proactive 
decisions by the user, or injects ( events ) that 
will dynamically modify the conditions within 
the simulation. The Primary Variable is 
composed of several Micro Variables (MVs). 
These MVs are assigned a weight of 
importance by the developer, and are 
modified directly by the simulation in order to 
determine the value of the Primary Variable. 

Resources'. Within the simulation the learner 
is responsible for optimally using and 
maintaining several resources towards the 
success of the mission. These resources 
include food, money, political capital, medical 
supplies. 


Win Game Scenario : If the user is properly 
engaged in the simulation and is generating 
positive results, the Primary Variables 
(villager relationship, soldier well-being, etc.) 
should all be increasing towards a success 
threshold. This threshold value for each of 
these variables will be built into the simulation 
where once all the variables have surpassed 
this value, the simulation is complete. The 
communicated reward for the leader is seeing 
all of the positive results in the scenario. 
Conversely, if all four variables fall below the 
minimum threshold, and remain there for a 
designated period of time, the leader is 
advised that the situation has deteriorated to 
the point where the scenario is unwinnable. 
They may restart the game and try again. 

5.0 DEVELOPING SCENARIOS 

Our approach to developing the scenarios 
was iterative and systematic, while leaving 
room for creativity and emotional verve. In 
all, 1 1 scenarios were developed. Initially, we 
developed a single, prototypical scenario to 
practice applying the theoretical principles 
discussed previously and to refine our 
process. Our first step was to write brief 
descriptions, about a paragraph in length, for 
each scenario. This allowed us to create an 
overarching storyline across all of the 
scenarios, bringing cohesion to the scenarios. 
These descriptions also served as a map for 
developing each of the scenarios, such that 
when developing an earlier scenario we could 
create decision options that would set up later 
scenarios. Within each scenario there were 
several details including dialogues between 
participants, immediate, long-term effects, 
multi-order decision impacts, resource 
modifications, and decision points for the 
learner. Table 1 below contains a short 
summation of the scenarios and competing 
values taking place in the simulation. 
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Table 1 : Example scenarios and competing values 


Scenario 

Summary 

Competing Values 

Meeting with 
Village Elders 

The overall objective is to make decisions about the initial allocation of 
resources. The leader is new to this assignment and this is the 
leader’s first meeting with all the elders together. 

This is a critical meeting for the leader because it will set the tone for 
his/her relationship with the local population. In addition, because the 
leader is new to this assignment, subordinate Soldiers are getting to 
know him/her as well. Consequences of decisions in this scenario will 
have long-term impacts because the relationships are just forming. 

• Ensuring there is adequate security 
and food for both villagers and 
refugees 

• Desire to make villagers happy and 
build a positive relationship vs. 
limited resources to allocate toward 
village 

• Building relationships with the 
leader of the refugee camp and the 
village 

First Attack 

The leader must determine the threat level against the Forward 
Operating Base (FOB) and decide how best to ensure its security 
while continuing to perform governance tasks in the village. Ultimately 
the leader must determine how many Soldiers to assign to security 
tasks and what type of security tasks they should be performing. 

• Assigning Soldiers to security tasks 
within the safety of the FOB; 

• Assign some Soldiers to ambush 
patrols outside the FOB versus 
allocate Soldiers to build 
infrastructure for the village. 

Captured Son 

Your Soldiers have just gotten in from an all-night patrol and are 
putting away their equipment. During the patrol, there was an ambush 
and subsequent firefight. One of your Soldiers was shot in the back. 
The medic does not know if he will walk again. Your Soldiers capture 
one of the teenage assailants who says he was coerced by the rebels 
into taking part in the ambush. However, one of your Soldiers is sure 
that the teenager had a weapon and was firing it. 

As your Soldiers are putting away equipment, one of the local leaders 
stops by to tell you that you have mistakenly taken his son prisoner 
and demands his immediate release. The local leader promises you 
that his son will never take part in a rebel army attack on US Forces 
again. It is the perfect opportunity to build goodwill with a local leader 
and have him in your debt, yet the very idea of letting go a potential 
rebel Army recruit go could send the wrong message. And your 
Soldiers would see you as too soft on security and too keen on 
“building goodwill” at their expense. Do you release the son or not? 

• Soldier morale vs. relationship 
building with the local leader 

• Taking a firm stance on security vs. 
building goodwill 

Security 

Dilemma 

The village leader is walking purposely through the village on his way 
to a meeting with the local police chief. The local police force is at 
about 50% of its strength but they were stripped of their weapons and 
equipment when the rebels initially swept through the area. He has 
requested that you re-arm his police force with weapons so that they 
can once again fulfill their role in maintaining law and order in the 
village. Yet some of the killed rebels you came across were partially 
clad in police uniforms and bearing police side arms. 

• Security of the village vs. rebuilding 
local policing capacity 

• Increasing security vs. building a 
relationship with the villagers 

Hiring Dilemma 

The village Water & Sanitation Director was killed when the rebels 
initially attacked the village and a new one must be hired. The council 
of elders would like to be in charge of finding the new Director, but you 
are not sure how quickly they can do this or whether expertise and 
experience will be the most important criterion used for selecting a 
new Director. Clean water and sanitation are key to the health of the 
village and preventing outbreaks of disease that could threaten 
everyone. The council assures you that they will find someone very 
good, but it doesn’t seem like the council has established a good 
process for working out disagreements (at least without outside help), 
and your observation has been that key positions are given out based 
more on relationships and tribal status than actual expertise. 
Consequently you are very concerned with how they will decide on 
someone to fill a high-level post. 

• Building a positive relationship with 
the villagers vs. building capacity of 
the village. 
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Farmer Loyalty 

You are given a situation report at the FOB that there is trouble 
brewing between your Soldiers and the local farmers. You decided to 
drive out to one of the farms, where a group of farmers has gathered 
to meet with you. 

A group of your Soldiers has been working with local farmers to try to 
get the farms back up and running and enhance irrigation systems in 
the area. While they are working with the farmers, they start realizing 
that the farmers are selling grain and food to the rebels. When they 
confronted the farmers about it, the farmers finally admit that the 
rebels are paying much more for the grain than they are able to get by 
selling supplies to the refugee camp aid workers, who you had thought 
were the major buyers. The farmers are also fearful of retribution if 
they refuse. 

• Building a relationship with the 
villagers vs. increasing security (by 
not allowing the farmers to supply 
the rebels) 

Refugee Crisis 

An NGO aid worker and their coalition security force say a new wave 
of refugees just showed up. She says apparently there is not enough 
shelter, medical supplies, food and water for all the refugees who have 
just arrived. Some riots and fights have broken out inside the fenced 
perimeter of the camp. Refugees are fighting over the limited shelter 
space and place in line for food and water. When you get there, you 
can see the aid workers trying to distribute the limited food supplies 
with panicked looks on their faces. They have now moved their truck 
outside the wire, tossing food bags over the fence to the fighting mob. 
Technically you do not have responsibility for the refugee camp, and 
the villagers are opposed to giving up resources for the refugees, but 
the refugees could present a threat to the NGO workers and Mubasi if 
they start foraging for food. 

• Gaining political capital with the 
village vs. responding to the 
humanitarian crisis at the refugee 
camp. 

• Gaining political capital with the 
village vs. intelligence gained from 
helping the refugees. 

Sick Cattle 

Cattle are getting sick at one of the biggest farms. It turns out they 
have been poisoned by toxic weeds leaves that have been spread 
around the grazing area. A significant number in the herd will need to 
be put down. Many are convinced that it’s the Lansi tribe members in 
the town (sympathetic to the rebels) who are at fault. The local police 
chief (who is Lansi) say he will lead the investigation. Local Mutsi 
elder objects - saying that the Lansi police chief is biased. The Mutsi 
want the U.S. to lead the investigation. However, if you don’t let the 
chief do the investigation, you will clearly be undermining his authority, 
and the villagers may go back to depending on you to settle disputes. 
Who do you put in charge of this investigation? 

• Building local capabilities vs. 
building a relationship with the 
villagers 

Flash Flood 

A flood did major damage to the water treatment plant. It also 
destroyed the bridge between the farmland to the east and Mubasi, 
virtually cutting the route by which farmers take their crops to market, 
and processed foodstuffs and supplies come to Mubasi. How will you 
allocate your resources to ensure safe drinking water, food, without 
compromising security? 

• Relationship with the villagers vs. 
security 

• Relationship with the villagers vs. 
Soldier morale 

Hiring Dilemma 
Part 2 

Sanitation has been a disaster (evidenced by trash everywhere and a 
growing rat population). Your senior medical personnel tell you that 
the village is at high risk for an infectious disease outbreak if the trash 
is not picked up and the rat population continues to grow. Soldiers 
could contract various diseases, as well. Furthermore, you just found 
out that many of the staff at the Sanitation Department have not yet 
been paid, but the money seems to have all been spent. The council is 
very nervous about removing the sanitation director because his family 
is very powerful and he is very popular. He insists that trash is not 
picked up and people have not been paid because his department is 
woefully underfunded. The council will not recommend that he be 
replaced. What do you do? 

• Building a relationship with the 
villagers vs. building infrastructure 

• Soldier well-being vs. relationship 
with the villagers 
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Figure 3: Mission at Mubasi Environment 


6.0 VIRTUAL ENVIRONMENT AND 
GAMEPLAY 

The Mission at Mubasi Decision-Making 
Simulation was developed using a 3D 
immersive web deployable gaming engine 
accredited for use on the Army Knowledge 
Online Portal. The virtual environment 
contained several indoor and outdoor areas 
(refugee camp, downtown, water treatment 
plant, FOB, farm, school house, and new 
construction areas) and over 400 virtual 
avatars that provided interaction and 
feedback to the learner while immersed in 
the gaming environment. As the learner 
progressed through the simulation, visual 
and audio cues were provided to help guide 
the progress on each of the primary 
variables. For instance if villager well-being 
was low, villagers would sit idle in the 
streets and less activity would be taking 
place in the market. 

7.0 BETA TEST 

A beta test of the Mission at Mubasi 
Decision-Making Simulation was held at an 
Army classroom with 16 participants in the 
range of senior non commissioned officers, 
and junior officers. Participants were given 


introductory information on how to use the 
simulation, and were then asked to play 
through eleven scenarios and present 
objective and subjective feedback 
responses to the Mission at Mubasi 
simulation. 

• Will this lesson be useful to you on the 
job? If so, how ? If not, why not? 

• Were the decision dilemmas 
significantly challenging? 

• Was it clear that decisions you made 
were tied to outcomes (1st and 2nd 
order effects) in the virtual world? 

• What aspects of the simulation did you 
find engaging or stood out as well done? 

• What, if anything, would you change 
about the simulation? 

• Please provide any additional comments 
you have about the lesson. 

8.0 RESULTS 

Beta test results were evaluated and 
categorized into two general categories, 
positive and negative impacts of simulation 
elements. Although there were many 
comments about the overall gameplay, 
Table 2 below presents the general 
responses we received from our two beta 
test groups. 
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Table 2: Responses from Beta-Test 


Simulation Element 

Positive Results 

Negative Results 

Understanding 
Competing Values 
(Quinn Model) 

Learners felt that the dilemmas 
presented were generally 
representative of their deployment 
experiences. 

Needed more information to 
determine best course of action 
(COA) 

Needed better understanding of 
story and mission to determine 
best COA. 

Understanding 
Complexity and 
Decision Making 
(Cynefin Model) 

Decisions were easily understood 
based on feedback within the 
simulation (visual cues, GUI 
elements) 

At times difficult to find patterns 
of behavior within simulation 


After Action Review helpful to 
understand impacts 


Engagement 

Experiences engaging and sometimes 
fun 

At times experiences were too 
complex to determine best 
approach 

Developing Strategies 

Interested in developing strategies 

Discrete outcomes minimized 
strategies 

Visual/Audio Cues 

Fun and engaging, strong affect in 
certain scenes (i.e. Attack at the 
Forward Operating Base) 

Issues with realism of soldier 
uniforms. 

Understanding After 
Action Review (AAR) 

Straight forward information provided 
with both graphical and text content 
describing the decision and impact 

Sometimes hard to understand 
multi-order impacts over time 
(concept can be mentally 
challenging). AAR didn’t go into 
enough detail about 
complicated topics. 

Navigation 

Simple game based interaction with 
engaging visuals and audio cues 

For non-gamers, sometimes 
difficult to understand 3D 
navigation and game User 
Interfaces. 



No Undo feature 


Competing Value Model: In terms of 
responding to the decision dilemmas, 
several participants felt that without enough 
information, especially in the early phases 
of the simulation, it was difficult to evaluate 
a best course of action. But the consensus 
was that it is unlikely a leader would always 
have all relevant information available to 
them in a real-life situation, and sometimes 
‘you had to do the best you could’. 


Multi-Order Impacts: Our second objective 
was to make sure that outcomes in the 
environment (multi-order effects) were 
clearly tied to decisions or groups of 
decisions. This was a difficult challenge as 
information in the simulation was coming 
from multiple sources; it was therefore 
decided that directing the learner towards 
specific cues would be very important in 
making sure they understood relationship 
between cause and effect. 


345 



9.0 CONCLUSION 

One of the most challenging aspects of 
developing a complex decision making 
simulation is to make a simulation real 
enough to simulate the variation in decision 
impacts, but simple enough where the 
learner is not lost in a myriad dynamics 
taking place in the game. The Mission at 
Mubasi simulation will continue to be 
evaluated over the course of the next 
several months for its learning efficacy. 
There is a great deal to understand about 
how decision-making can be trained 
including a) which are the most appropriate 
decision making models?, b) how important 
and to what level of fidelity should decisions 
be modeled within a simulation?, and c) 
what metrics are necessary to determine 
how well the learner was able to understand 
decision making impacts? 

A great deal of work within the simulation 
was designed to challenge the user to 
recognize cues in the environment beyond 
traditional analysis of cause and effect. The 
emergent patterns (unordered domain) were 
designed to suggest to the learner that 
multiple decisions that were weighted 
towards any one of the variables could 
jeopardize the end result of the game. For 
example, if the learner were to make well- 
being or security, eventually this would 
cause events in the simulation that would 
decrease the overall success of the mission. 
Data collected from our users showed that 
emergent pattern recognition was most 
helpful when cues were ample, relevant, 
and consistent in the simulation. If cues 
were occurring in simulation that did not 
appear to coincide with decisions, it was 
more difficult for the learner to recognize the 
decision impact. The instructional designers 
used user interface cues to help clarify 
multi-order effects. For example, animating 
energy bars on the user interface helped to 
focus the attention of the learner on global 
changes in the environment, rather than 
direct impacts based on immediate 
decisions. 


Future versions of the decision-making 
simulation will incorporate a schema format 
for extracting metrics for evaluation within 
the After Action Review (AAR) process. 
Additionally, it is our intention to continue to 
refine tools for experts to incorporate 
decision-making exemplars into the 
simulation framework. 
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